System and method for determining an orientation and position of an object

ABSTRACT

A system includes a computation device having an input module adapted to receive data defining a single two dimensional image, an image analyzing module configured to receive the data and analyze the single two dimensional image to determine a two dimensional orientation representative of a three dimensional orientation and position, a position calculating module configured to receive the two dimensional orientation from the image analyzing module and determine the three dimensional orientation and position of the object, and an output module adapted to send information relating to the three dimensional orientation and position of the object.

CROSS-REFERENCE TO RELATED APPLICATION

The present application is a continuation of U.S. patent applicationSer. No. 12/642,625, filed Dec. 18, 2009, entitled SYSTEM AND METHOD FORDETERMINING AN ORIENTATION AND POSITION OF AN OBJECT, which applicationclaims priority to U.S. Provisional Application No. 61/203,246, filedDec. 19, 2008, entitled TECHNIQUES FOR AUTONOMOUS VEHICLE CONTROL. Thepresent application is also related to U.S. patent application Ser. No.12/642,605, filed on Dec. 18, 2009 entitled SYSTEM AND METHOD FORAUTONOMOUS VEHICLE CONTROL. The contents of both of the above referencedapplications are hereby incorporated by reference herein in theirentireties.

TECHNICAL FIELD

The present disclosure relates to systems and methods for determining anorientation and position of an object. In particular, the presentdisclosure relates to systems and methods for determining the threedimensional orientation and position based on a two dimensionrepresentation of the object.

BACKGROUND

The determination of a three dimensional position and orientation of anobject can be a complicated mathematical task. Current methods forperforming this task can include, for example, using Euler's angles in amatrix to represent the entire object at once. This approach can includea large amount of ambiguity due to multiple possible solutions. In somecases, this approach can leave up to sixteen possible solutions and onlyone of the solutions may be correct.

In addition to the result being ambiguous, processing the solution ofsuch a large problem can require a correspondingly large amount ofprocessing power. This processing power can be provided by a substantialcomputing device requiring substantial electrical power. These elements,while becoming more readily available are often inherently heavy, whichcan be a hindrance or even a preventative property in some applications.

SUMMARY

In one embodiment, a system for determining an orientation and positionof an object can include a computation device having an input moduleadapted to receive data defining a two dimensional image, an imageanalyzing module configured to receive the data and analyze the twodimensional image to determine a two dimensional orientationrepresentative of a three dimensional orientation and position, aposition calculating module configured to receive the two dimensionalorientation from the image analyzing module and determine the threedimensional orientation and position of the object, and an output moduleadapted to send information relating to the three dimensionalorientation and position of the object. In some embodiments, the imageanalyzing module can include a background subtraction componentconfigured to reduce interference associated with the data defining thetwo dimensional image. The image analyzing module can also include athreshold image component configured to create a threshold image frombrightest pixels and a component labeler configured to assigncoordinates to selected portions of the two dimensional image. The imageanalyzing component can also include a centroid calculating componentconfigured to define a centroid of one or more pixels of the image.

In another embodiment of the system for determining an orientation andposition of an object, the position calculating module can include aprocessing component configured to process a series of linear equationsto determine the three dimensional orientation and position of theobject. In some embodiments, the series of linear equations is a Taylorseries of equations. In still other embodiments, the positioncalculating module further comprises an assumption application componentconfigured to apply boundary conditions to the series of linearequations thereby simplifying the processing thereof.

In another embodiment of the system for determining an orientation andposition of an object, the system may also include a situational datacomponent accessible by the position calculating module, whereinsituational data is stored for use in simplifying the determination ofthe three dimensional orientation and position. In some embodiments, thesituational data can include boundary assumptions relating to the rangeof expected orientations of the object defined by a range of rotationangles about axes passing through the center of mass of the object. Theexpected orientations can relate to operational limits and conditions ofthe object and, for example, the range of rotation angles can include arange of angles about a longitudinal direction of travel of the object.In another example, the range of rotation angles can include a range ofangles about one or more directions transverse to the direction oftravel. In some embodiments, the situational data can be adjustablebased on the object and conditions for which the orientation andposition are being determined. In still other embodiments, thesituational data can include relationship information between the objectand a position indicator associated with the object.

In another embodiment of the system for determining an orientation andposition of an object, the computation device can further include animage capturing module configured to control an image detection device.The image capturing module can include an initial detection componentconfigured to activate a detection device, a timing component configuredto control the frequency of detection device actuation, and a shut downcomponent configured to deactivate the detection device.

In another embodiment, a method for determining an orientation andposition of an object can include receiving image data and storing theimage data in a computer readable storage medium, the image dataincluding a two dimensional depiction of the object, and using acomputation device having one or more modules for accessing the imagedata and determining the orientation and position of the object. Thedetermining can include analyzing the image data to determine a twodimensional orientation that is representative of a three dimensionalposition and orientation of the object and performing a threedimensional analysis limited by boundary conditions to determine thethree dimensional orientation and position of the object. In someembodiments, performing a three dimensional analysis can includeprocessing a system of linear equations, such as a Taylor series.

In another embodiment, the method of determining an orientation andposition of an object can include applying boundary conditions to limitthe variables associated with the three dimensional position andorientation of the object, wherein the boundary conditions relate to therange of expected orientations of the object defined by a range ofrotation angles about axes passing through the center of mass of theobject. In some embodiments, the expected orientations can relate tooperational limits and conditions of the object. For example, the rangeof rotation angles can include a range of angles about a longitudinaldirection of travel of the object. In another example, the range ofrotation angles can include a range of angles about one or moredirections transverse to the direction of travel. In still otherembodiments, the boundary conditions can relate to a known relationshipbetween the position indicator orientation and the object orientation.

In another embodiment, analyzing the image data can include labeling oneor more portions of the image data with two dimensional coordinates. Inother embodiments, analyzing the image data can include filteringbackground noise out of the two dimensional representation and creatinga threshold image.

BRIEF DESCRIPTION OF THE FIGURES

While the specification concludes with claims particularly pointing outand distinctly claiming the subject matter that is regarded as formingthe various embodiments of the present disclosure, it is believed thatthe embodiments will be better understood from the following descriptiontaken in conjunction with the accompanying Figures.

FIG. 1 depicts a method of determining the orientation and position ofan object according to certain embodiments;

FIG. 2 depicts a method of analyzing an image according to certainembodiments;

FIG. 3 is an exemplary diagram depicting a method for vehicle control;

FIG. 4 is a perspective diagram of a system for vehicle controlaccording to certain embodiments;

FIG. 5 is a close-up diagram of a first sub-system of the system of FIG.4 on the vehicle of FIG. 4;

FIG. 6 is a schematic diagram of a second sub-system of the system ofFIG. 4;

FIG. 7 is side view diagram of the system of FIG. 4 showing theinteraction of the first and second sub-systems;

FIG. 8 depicts a display of a detection system of the system of FIG. 4;

FIG. 9 depicts a display of a detection system of the system of FIG. 4;

FIG. 10 is a detailed view of the system of FIG. 4;

FIG. 11 is a diagram reflecting a method of autonomous vehicle control;

FIGS. 12 a-12 d are exemplary images captured by a detection device ofthe system of FIG. 4;

FIG. 13 is a graph of the relative intensity of certain sources ofelectromagnetic radiation;

FIG. 14 a is an exemplary second sub-system for use with the system ofFIG. 4;

FIG. 14 b is another exemplary second sub-system for use with the systemof FIG. 4;

FIG. 14 c depicts modules of the system according to an embodiment;

FIG. 15 is a schematic diagram of a first and second subsystem of thesystem of FIG. 4.

DETAILED DESCRIPTION

The present disclosure relates to systems and methods for determining anorientation and position of an object. In a particular embodiment, thepresent disclosure relates to systems and methods for determining thethree dimensional orientation and position based on a two dimensionrepresentation of the object. For example, three points on a twodimensional plane can be used together with knowledge about therelationship of the configuration of the points to the orientation ofthe object and knowledge of the operational limits of the object tocalculate the objects three dimensional orientation and position.

In particular, boundary assumptions based on the conditions andoperational limits of an object can be used to simplify the threedimensional problem. For example, with reference to FIG. 1, theseassumptions can allow for decoupling the problem into three orthogonalplanes, calculating angles of the object's position on one or more ofthese planes, and combining the results to achieve a three dimensionalorientation and position. With reference to FIG. 2, the two dimensionalrepresentation of the object can include a two dimensional image andadditional measures can be included to analyze the image and obtain theinitial two dimensional orientation that is representative of the threedimensional orientation and position.

In some embodiments, the above described system and method can be usedfor autonomous vehicle control. For example, referring to FIG. 3, avehicle can be located at a first coordinate A (x1, y1, z1) in threedimensional space. The system in accordance with the present disclosurecan be used to localize the vehicle along the arrow C to a secondcoordinate B (x2, y2, z2).

Accordingly, a relatively lengthy discussion of exemplaryimplementations of the above mentioned systems and methods will followto provide some context for further discussion of the these systems. Assuch, the description of FIGS. 3-14 b is offered in support of the moredetailed discussion of these systems and methods with regard to FIGS. 14c, 1, and 2 provided later in the specification. It is noted that thesystem and method described herein is not limited to the exemplaryimplementations shown and described below. That is, use with autonomousvehicles is just one arena within which the systems and methods can beused. Other arenas may include analysis of stationary or moving objectsother than vehicles or autonomous vehicles. Additionally, while much ofthe below discussion relates to a detection device being remote from thevehicle with the vehicle approaching the detection device, otherembodiments can include a detection device on the vehicle allowing thevehicle to sense and avoid objects. Other implementations will beapparent to those of skill in the art.

An autonomous vehicle capable of use with the systems and methods of thepresent disclosure may include, but is not limited to, an autonomousunderwater vehicle, an unmanned ground vehicle, or an unmanned aerialvehicle. It is to be noted that autonomous control can be provided withor without the presence of an onboard operator or remote operator. Assuch, the disclosed system is not limited to unmanned vehicles or thosewithout remote control capabilities.

Referring to FIG. 4, the use of the system with an unmanned aerialvehicle (UAV) is shown. An environment 110 is depicted including avehicle 100 in a flying configuration at an altitude above the ground113 equal to the distance C. The system of the present disclosure maycause the vehicle 100 to localize approximately along the path A (andapproximately along the over-ground projection B) to a target area Dlocated, for example, on the ground 113, or at any other position, suchas on a tripod, on a capture net, on a moving vehicle, on top of abuilding, aerial refueling craft, docking space, etc. Thus, inlocalizing to the target area D, the UAV together with the system hereindescribed may be capable of performing an autonomous, feedbackcontrolled approach to landing.

Referring still to FIG. 4, the system may generally include twosub-systems: a first sub-system 120 located on and integrated within thevehicle 100, and a second sub-system 140 independent of the vehicle 100and located at a fixed position relative to the vehicle 100, for exampleat or near position D. The first sub-system 120 may comprise componentswhich receive information, process that information, and control thedirection of the vehicle 100 in two or more dimensions. Such componentsmay be referred to as an “autopilot.” The first sub-system 120 may alsocomprise components which allow the vehicle's position to be detected.The second sub-system 140, which may be located at or near target areaD, may comprise components for detecting the position of the vehicle100, which may work in cooperation with the components of the firstsub-system 120 which allow the vehicle's position to be detected. Thesecond sub-system 140 may also comprise components that process thevehicles position and transmission components that allow the secondsub-system 140 to transmit information to the directional controlcomponents of the first sub-system 120. Thus, the autonomous feedbackcontrolled localization may function generally in the following manner.The components of the second sub-system 140 may detect the position ofthe vehicle 100. The position may be processed to determine thevehicle's current position relative to the target area and the secondsub-system 140 may transmit positional or control information to thefirst sub-system 120 located on the vehicle 100. The first sub-system120 may then receive the transmitted information, process theinformation as required, and control the vehicle 100 so as to localizeto the target area D.

Referring now to FIG. 5, particular attention can be drawn to thecomponents of the first sub-system 120. The first sub-system 120 caninclude a receiver 122, an inertial measurement unit (IMU), acomputation device 124, a control system 126, and a position indicator128. In some embodiments, the computation device 124 may or may not beprovided. For example, depending on the nature of the system, the firstsub-system 120 may be adapted to develop control information based on aposition provided by the second sub-system 140. In this system, acomputation device 124 may be provided to develop this controlinformation. In other embodiments, this control information may bedeveloped by the second sub-system 140 and the computation device 124can be omitted. Additionally, the IMU 123 may be operatively coupled tothe computation device 124. The IMU may include, but is not limited to,any of the following components alone or in combination: gryos,accelerometers, magnetometers, global positioning system (GPS),barometer, thermometer, thermocouple, or alpha beta sensor, etc.

The receiver 122 can be positioned on the vehicle 100 and can beconfigured to receive a signal from the second sub-system 140. Thesignal may carry control instructions or positional informationdeveloped or obtained, respectively, by the second sub-system 140 andtransmitted thereby. The receiver 122 can thus be configured to filterthe control instructions or the positional information from the signal.In the case of control instructions, the receiver 122 can further beconfigured to communicate the instructions to the control system 126. Inthe case of positional information, the receiver can further beconfigured to communicate the instructions to the computation device124. The receiver may be any known receiver capable of receiving asignal, filtering the signal, and communicating the information carriedby the signal to another device. In one embodiment, the receiver 122 aradio receiver adapted to receive radio wave transmissions with digitalor analog information relating to a vehicle's position or control. Thecomputation device 124 may receive additional measurement informationfrom the IMU 123.

The control system 126 can also be positioned on the vehicle 100 and canbe operably connected to the directional controls of the vehicle 100.The directional controls to which the control system 126 is connecteddepend on the type of vehicle 100 being employed. The control system 126may be configured to control the vehicle's motion in one or moredimensions. For example, in the case of an autonomous underwatervehicle, the control system 126 may be operably connected to the fins,rudder, and propulsion system in order to control the vehicle's depth,lateral position, and forward position in the water. In the case of anunmanned ground vehicle, the control system 126 may be connected to theaccelerator/decelerator and steering mechanism to control the vehicle'sforward position and lateral position over the ground. Furthermore, inthe case of an unmanned aerial vehicle, the control system 126 may beconnected to the engine and the flight control surfaces, in order tocontrol the vehicle's altitude, and lateral and forward positions. Acontrol system 126 may be connected to other types of vehicles in likemanners to control such vehicles' motion in one or more dimensions.

In one embodiment, as shown in FIG. 5, the control system 126 may beconnected to the directional control components of a UAV. Suchdirectional control components may include, the elevator 102, rudder104, ailerons 106, and powerplant 108 (e.g., reciprocating piston,turbofan, etc.). The control system 126 may preferably be configured tofully control all aspects of the UAV's directional movement, includingcontrolling full range of motion of the elevator 102, rudder 104, andailerons 106, and full throttling of the powerplant 108 from idle tofull throttle. The individual mechanized components of such a controlsystem 126 will be known to and appreciated by those skilled in the art,and may include, for example, actuator/cable assemblies, servos,hydraulics and air/fuel mixture regulators, among others.

The first sub-system 120 may further include a computation device 124 tocompute control instructions for the control system 126 to use tocontrol the movement of the directional control components of the UAV tocause the UAV to fly in a desired manner. For example, such acomputation device 124 may provide control instructions to the controlsystem 126 to cause the UAV to fly from a first, known position to asecond, desired position through appropriate manipulation of thedirectional control components. Thus, if the computation device 124receives known positional information of the UAV which is, for purposesof illustration, below and to the left of (relative to the direction offlight) a desired position, the computation device 124 may developinstructions such that the control system 126 causes the elevator 102 todeflect upwardly (thereby causing the UAV to gain altitude), thepowerplant 108 to increase output (thereby causing the UAV to maintainadequate airspeed during a climb), the left aileron 106 to deflectdownwardly and the right aileron 106 to deflect upwardly (therebycausing the UAV to bank to the right), and the rudder 104 to deflect tothe right (thereby counteracting adverse yaw caused by the banking andpossibly the induced p-factor in the case of a propeller driven UAV).Such positional information may be augmented/validated by themeasurements made by the IMU 123 and sent to the computation device 124.The magnitude of such positional control inputs by the control system126 may be determined by the relative distance between the knownposition and the desired position, among other factors. Furtherpositional information received by the computation device 124 may causefurther changes to the directional control components, again based onthe UAV's known position relative to a desired position.

With continued reference to FIG. 5, the first sub-system may furtherinclude one or more position indicators 128 located on the vehicle 100.A position indicator 128 may include an element, detail, surface schemeor other indicating feature adapted to mark a point on the vehicle 100.Any number of position indicators can be provided. Preferably three ormore are provided. In the shown, the position indicators 128 are in theform of three discreet point sources of electromagnetic radiationlocated at three points on the vehicle 100. The electromagneticradiation emitted by these indicators 128 may include, but is notlimited to, radio waves, microwaves, terahertz radiation, infraredradiation, visible light, ultraviolet radiation, X-rays and gamma rays.Point sources of electromagnetic radiation may be generated on thevehicle in any known manner. For example, LED lights may emit pointsources of visible light of any color. In some embodiments, a pointsource may be a reflection of electromagnetic radiation. For example,reflectors or reflective tape may be positioned on the exterior of thevehicle 100 causing sunlight to be reflected at those points. Or moresimply, the point sources may be known, discreet positions along theexterior of the vehicle which reflect sunlight and thereby provide apoint source of electromagnetic radiation of visible light in the colorof that point on the exterior of the vehicle 100. In one embodiment, aparticular paint pattern can be used to define the point sources. Instill another embodiment, reflective paint may be used such as paintwith metal flecks or other reflective materials included in the paint.The vehicle 100 shown in FIG. 5 includes three position indicators 128in the form of three point sources of electromagetic radiation disposedabout the exterior of the vehicle, although it will be appreciated thatgreater than three point sources may also be used.

In one embodiment of the presently disclosed system, the positionindicator 128 can be in the form of an LED-type point sources of aparticular wavelength. The point sources can be provided at any positionon the exterior of the vehicle 100. In one embodiment, the point sourcescan be provided at the greatest possible distances separated from thecenter of gravity (CG) of the vehicle 100. For example, as shown in FIG.5, point sources may be provided on each wingtip, and on the top orbottom side of the vertical stabilizer of the UAV. Alternatively, pointsources may be provided on each wingtip, and on the front of the UAV'snose, so as to avoid occlusion of the light source, which may occurdepending on the configuration and orientation of the UAV. LED's orother power requiring point sources may be connected to the internalbattery or other power source of the vehicle 100. Alternatively, eachpoint source may have its own power source, e.g., battery.

Referring now to FIG. 6, general reference will now be made to thepreviously mentioned second sub-system 140 of the system of the presentdisclosure. The second sub-system 140 may include a detection device142, a computational device 144, and a transmitter 146. The detectiondevice 142 may be adapted to detect the position and orientation of avehicle 100, while the computational device 144 may be adapted tointerpret the information from the detection device 142 and determinethe orientation and position as well as develop control instructions.The positional information and/or the control instructions can then betransmitted via the transmitter 146 to the first sub-system 120.

In one embodiment, the detection device 142 can be adapted to detect theposition indicators 128 of the first sub-system. In a particularembodiment, the detection device 142 can be an electromagnetic radiationdetection device. In this embodiment, the detection device 142 can beconfigured to detect the position of three or more point sources ofelectromagnetic radiation in two dimensions. The device 142 may beconfigured to provide a two dimensional display of the detected positionindicators 128. The two dimensional display may thereby show thedetected two-dimensional positions of the detected position indicators128 relative to one another. In one embodiment, the detection device 142may be a camera configured to detect visible light of a particularwavelength. For example, the camera may be adapted to detect theparticular wavelength or range of wavelengths generated by theparticular position indicators 128 provided on the vehicle. Moreparticularly, the camera may be adapted to detect the wavelengthgenerated by LEDs. Alternatively, the camera may be adapted withsoftware or firmware to only detect a particular frequency of flashinglight, wherein the light sources may utilize frequency modulation toprovide the desired frequency. For example, a camera with a fixed andsteady frame rate may be employed of detected light source flashing at afrequency of ⅛, ⅓, ¼, ½, etc, the frame rate of the camera.

Referring to FIG. 7, a detection device 142 is depicted as detecting theelectromagnetic radiation from the position indicators 128 located onthe exterior of the vehicle 100. As shown therein, the vehicle 100 maygenerally be localizing toward the detection device 142 in the directionA, and in this configuration, the position indicators 128 may generallybe detectable (i.e., not obstructed by other parts of the vehicle) bythe detection device 142.

Referring to FIG. 8, a two dimensional display 148 of the detectiondevice is shown. The image shown on the display 148 is what may appearfrom the configuration depicted in FIG. 7. Specifically, positionindicators 128 are depicted on the two-dimensional display 148 as thedetection device 142 detects their position in two dimensions. FIG. 8depicts a ghost image of the vehicle 100 in relation to the detectedposition indicators 128. Referring to FIG. 9, the two dimensionaldisplay 148 has been filtered to show only the position indicators 128and not the ghost image.

Referring again to FIG. 7, in one embodiment as adapted for use with aUAV, the detection device 142 may be located at any position, forexample, on a tripod, on a capture net, on a moving vehicle, on top of abuilding, aerial refueling craft, docking space, etc., and at anydistance from the flight path (arrow A) of the UAV. Preferably, thedetection device 142 may be located on the ground near a landing area150 where the UAV is desired to be landed. The detection device maydisplay the detected location of the radiation in two dimensions.Preferably, the first dimension is the azimuth of the source relative tothe horizon, and the second dimension is the altitude of the sourcerelative to the horizon.

Referring again to FIG. 6, the second sub-system 140 may further includea computer 144, or other computation device capable of performingmathematical calculations. The computer 144 may be operably connected tothe detection device 142, and may be configured to receive thetwo-dimensional display/data of the positional indicators 128 generatedby the detection device 142. In particular, a computer 144 in accordancewith the present disclosure may have encoded instructions thereonconfigured to calculate the two or three dimensional position of thepositional indicators 128 relative to the detection device 142, based onthe two dimensional display 148 generated by the detection device 142and further based on the known configuration of the position indicators128 on the vehicle 100. For example, in the case where the positionindicators 128 are three point sources on the exterior of an autonomousvehicle, the computer 144 may have information stored thereon related tothe position of the point sources on the exterior of the vehicle, andmay use that information to calculate the vehicle's two or threedimensional position relative to the detection device 142 based on thetwo-dimensional display 148 of the three point sources generated by thedetection device 142. Such calculation may be accomplished by any knownmathematical method, or approximation thereof.

The second sub-system 140 may further include a transmitter 146 operablyconnected to the computer 144. The transmitter 146 may be configured totransmit control instructions or position information related to thecontrol or position of the vehicle 100. The control instructions orposition information to be transmitted may be based on the computed twoor three dimensional position of the vehicle 100, as computed from thetwo-dimensional display 148 of the position indicators 128. Inparticular, control information may be transmitted based on thevehicle's current position in relation to a desired position. Thetransmitter 146 may transmit in a manner, for example radio waves, suchthat the transmission is receivable by the receiver 122 of the firstsub-system 120, located on the vehicle 100.

In one embodiment of the presently disclosed system for use with UAVs,as depicted in FIG. 9, a computer 144 may be operably connected todetection device 142 to mathematically transform the two dimensionalpositional information of a UAV into three dimensional positionalinformation (the third dimension being the distance of the UAV from thedetection device, or range), using the known positioning of the positionindicators 128 on the UAV. This calculation may be performed usingmathematical formulae. Preferably, the calculation is carried out usinglinear approximations.

A transmitter 146 operably connected to the computer 144 may beconfigured to transmit positional information (arrow B) so as to bereceivable by the receiver 122 (shown in FIG. 5) of the first sub-system120 on the UAV, as previously discussed. Alternatively, the transmitter144 may be configured to transmit control information to the receiver122. Where control information is to be transmitted, the computer 144may use the calculated position of the UAV as compared to the desiredposition of the UAV to transmit control information to the UAV to causethe UAV to fly toward the desired position (along arrow A), as alsodiscussed above. The instructions here can be the same or similar tothose instructions provided by the computation device 124 described withrespect to the first sub-system 120 for the condition where the firstsub-system received only positional information from the secondsub-system 140.

As previously mentioned, the present disclosure relates to systems andmethods for automated, feedback-controlled localization of a vehicle toa point in two or three dimensions. A method in accordance with thepresent disclosure may include detecting the position indicators 128,displaying the position indicators 128 on a two-dimensional display,calculating the two or three dimensional position of the vehicle 100based on the detected position indicators and on the known configurationof the indicators 128 on the vehicle, developing control information,transmitting positional or control information to the vehicle based onits calculated position relative to a desired position, receiving thepositional or control information, developing control information asrequired, and adjusting the vehicle controls to cause the vehicle tolocalize to a point, based on the positional or control informationreceived.

Accordingly, embodiments of the presently described method may beadapted for use with a UAV to cause the UAV to localize to a desiredposition in three-dimensional space, for example, an autonomous feedbackcontrolled approach to landing. As depicted in FIG. 10, such a methodmay include, for example, detecting the position indicators 128(reference numeral 10), displaying in two dimensions the detectedpositions of the point sources of the UAV on a display (referencenumeral 11), computing the three-dimensional position of the UAV basedon the detected two dimensional position of the point source and furtherbased on the location of the point sources on the exterior of the UAV(reference numeral 12), developing control instructions (referencenumeral 13), transmitting position or control information to the UAV(reference numeral 14), and manipulating the directional controlcomponents of the UAV to cause the UAV to fly to a desired position(reference numeral 15). As noted by the circular nature of the FIG.,this process can occur in a looped fashion to repeatedly capture theposition of the UAV and repeatedly control its path of flight.

With specific attention to the procedures of the method outlined above,detecting the position indicators 128 may include capturing a visualimage of the vehicle 100 including the position indicators 128. Thedevice may provide a two dimensional display 148 of the detectedposition of the sources of electromagnetic radiation relative to oneanother. Providing this display may include portraying the positions ona viewable screen or it may include merely creating an electronic recordof the positions of the position indicators 128 in a two dimensionalplane.

Based on these relative positions of the point sources on the twodimensional display, the two or three dimensional position of thevehicle 100 relative to the detection device 142 may be calculated. Thecalculation may include the known position of the position indicators128 on the vehicle 100. A computer, or other computation device, 144connected to the detection device 142 may compute using the twodimensional information generated from the device 142 with the positionindicator 128 position information to provide a two or three dimensionalposition of the vehicle 100 relative to the detection device 142. Thecomputation may be done by any mathematical technique. Such mathematicaltechniques may include, for example, a series of two or three linearapproximations, for example, Taylor series expansions.

Depending on the nature of the system, the computer 144 may alsocalculate control instructions. That is, where the system is set up toprovide control instructions to the first sub-system 120 in lieu ofmerely positional information, the computer 144 can further calculatecontrol instructions. This calculation can include a comparison of theposition of the vehicle 100 as compared to the desired position and canfurther include developing vehicle commands for adjusting the trajectoryof the vehicle 100.

Based on the calculated position of the vehicle 100 relative to thedetection device 142, a transmitter connected to the computer orcomputation device 144 may transmit position or control information tothe vehicle 100. This transmission can occur via radio transmission orother transmission capable of carrying the position or controlinformation. A receiver 122 on the vehicle 100 may be configured toreceive such position or control information. The receiver 122 may beoperably connected to a control system 126 on the vehicle 100, thecontrol system 126 being capable of controlling all of the directionalcontrol components of the vehicle 100. The vehicle 100 may use thispositional or control information to localize to a desired point orlocation. In one embodiment, the vehicle 100 may localize to theposition of the detection device 142.

In the case of positional information being transmitted to the vehicle100, the computation device 124 may determine whether the vehicle 100 islocalizing to the desired point based on the vehicle's change inposition over time. If the computation device 124 determines that thevehicle is proceeding on a path to the desired point, then the systemmay not implement any directional control changes. If, alternatively,the computation device 124 determines that the vehicle 100 is deviatingfrom the localizing course, appropriate directional control changes maybe input to the vehicle's directional control components to cause thevehicle 100 to localize to the desired point or position.

As mentioned above, in the case of control information being transmittedto the vehicle control system, the computer or computation device 144 ofthe second sub-system 120 may first determine whether the vehicle 100 isproperly localizing to the desired point based on its calculated changesin position over time, and then directly transmit directional controlinformation to the vehicle 100, if needed. Upon receiving such controlinformation, the control system 126 on the vehicle 100 may causedirectional control changes to be input to the directional controlcomponents of the vehicle 100.

Referring to FIG. 10, in one embodiment of the method of the presentdisclosure suitable for use with a UAV, the detection device 142 may bepositioned at or near a desired landing area 150 for the UAV. Thecomputed position of the UAV, based on the detected position of thepoints sources of electromagnetic radiation and the known configurationof the point sources on the exterior of the UAV, may be used to localizethe UAV (along line A) to the position of the detection device 142,thereby causing the UAV to localize to the landing area 150 as anapproach for landing. Position or control information may be transmitted(via transmitter 146) from the detection device 142/computer 144 tocause the control system 126 on the UAV to manipulate the directionalcontrols of the UAV to maintain a desired approach course (includinghorizontal (azimuthal) position, glideslope (altitude), and airspeed) tothe landing area 150.

Certain components of the system and method of the present disclosurewill now be described in greater detail with regard to preferredembodiments adapted for use with a UAV. As will be appreciated by thoseof skill in the art, other components may be used interchangeablywithout departing from the spirit and scope of the disclosure, as setforth in the appended claims—thus, the following example embodiments arenot in any way intended to be limiting.

In one embodiment, position indicators 128 may include point sources inthe form of LEDs. LEDs are desirable because they are lightweight,durable, have a high output to power ratio, have a wide viewing angle,and are distinguishable against background noise for purposes ofdetection. FIGS. 12 a-d depict the detected electromagnetic radiationfrom LEDs positioned on a UAV at a distance of approximately 500 feet,wherein the UAV is oriented, respectively, at 60, 45, 30, and 0 degreesrelative to a detection device configured to detect LED light. The leftLED in each Figure has a power of 1 Watt, while the right LED in eachFigure has a power of 3 Watts, although any Wattage of LED may be used.Desirable wavelengths of electromagnetic radiation (LED light) may be inthe approximately 635-808 nanometer range, which is largelydistinguishable against sunlight, blue skies, and red sunsets. FIG. 13shows a graph of relative intensities of background visible light causedby the sky and the sun, which may be used in selecting an appropriatewavelength of LED for use with the teachings of the present disclosure.Other suitable sources of electromagnetic radiation in the visiblespectrum may include, but are not limited to, incandescent, fluorescent,arc lamp, and gas discharge lighting.

With regard to the detection device 142, some embodiments in accordancewith the present disclosure may employ a camera sensor. Desirablecharacteristics of a detection device 142 may include, but are notlimited to, frame rate, resolution, and range of wavelength detectioncapabilities. For example, the Micron® MTV9032 camera sensor has beenfound to be desirable for use in detecting LED light in theapproximately 635-808 nanometer range, as discussed above, or morebroadly in the approximately 375-900 nanometer range. This particularmodel has the benefits of a high frame rate (60 frames per second) and ahigh resolution (2.4 megapixel) to more accurately determine and displaythe position of the point sources of electromagnetic radiation forsubsequent positional calculations. Other camera types and styles can beused. The device may further be configured with a variety of lenses.Appropriate lens selection may be determined by the environment in whichthe system is being used. For example, some applications may require along focal length (for example, where detecting the UAV at long distanceis desirable); alternatively, some applications may require a wideviewing window or horizon length (for example, where detecting the UAVacross a broad range along the horizon is desirable). To determine fieldof view and focal length, the following equations may be used. Withrespect to the field of view:

$\theta_{FOV} = {2\arctan \frac{DesiredHorizonLength}{2D}}$

Wherein D is the distance to the UAV. With respect to the focal length:

${{Focal}\mspace{14mu} {Length}} = \frac{C}{\tan \left( \theta_{FOV} \right)}$

Wherein C is the aperture number of the electromagnetic radiationdetection device, which in some embodiments may be a camera. Inembodiments where the UAV is desired to be viewable at 500 feet, anapproximately 22 degree window and 60 meter horizon length may be used,which equates according to the above equations to an approximately 12-13millimeter focal length and 8 as the aperture number.

Furthermore, the detection device 142 may be outfitted with anappropriate light (optical) filter, for example, a band pass filter, tofurther enable the device to more accurately detect the position of theLEDs, and reduce the background “noise” which may be particularlyprevalent on sunny days. Such an optical filter may be a narrow bandpass filter which allows the specific frequency of LED light to passthrough while attenuating others. In one embodiment employing 635 nmLEDs, a band pass filter with a 10 nm pass may be used. Preferably, aband pass filter will not attenuate the pass band at all—however, if asharp attenuation of wavelengths outside the band is desired, a bandpass filter which attenuates the pass band up to 60% or more may beused. In alternative embodiments, electromagnetic radiation outside thevisible spectrum may be employed to avoid visible light backgroundnoise.

With regard to the computer or computation device 144, as depicted inFIG. 14 a, the computation device 144 may comprise a processing board 31having included thereon an electromagnetic radiation detector (sensor)port 24 for receiving information from the electromagnetic radiationdetector 22 through cable 23, a signal converter 25 for converting thetwo dimensional display from the detector into an electronic signal, RAM26, a processor 27 for performing the position and/or controlcalculations, a signal converter 28 for converting the positional orcontrol information into a transmittable signal, memory 29, a radiocontroller transmitter port 30 for communicating positional/controlinformation, via cable 32, to the radio controller 33 and transmitterantenna 34. In some embodiments, the memory 29 can be in the form ofprogram memory. Desirable qualities of a computation system may includea high frequency processing rate and large memory capacity, due to thelarge amount of data being sent from the detection device.

In particular, the Analog Devices® Blackfin Dual DSP chip (BF561) hasbeen determined to be a suitable computation device for use with thepresently disclosed systems and methods. In particular, this deviceachieves a high computation rate, which aides the speed with whichpositional or control information may be transmitted to the UAV afterdetecting the position indicators 128. Programming of the computationdevice may be done in any computer language, with VisualDSP++4.5 being apreferred language. Using this particular example computation device,the image may be captured by the detection device 142 and transferred tothe processing board 31 using a parallel data bus running at 27 MHz. TheBF561 may read in the frame data through its Parallel Port Interface(PPI), PPIO. The frame data may be transferred via Direct Memory Access(DMA) to Level 3 (L3) SDRAM, which has 64 MB divided into four banks.Core A of the BF561 may handle the PPIO interrupt routine, which simplysignals that a frame has been successfully captured. Core A may alsohandle in its main function, which consists of an infinite loop, thebuffering scheme to place input frames into one of two frame buffers.

Using frame buffers in separate memory banks may benefit the processingspeed because of the nature of the DMA channels and SDRAM memory access.SDRAM memory access may experience increased latencies if simultaneousDMA transfers are initiated on the same bank. Further, if DMA transfersare initialized on the DMA channels, latencies may increase. In oneembodiment, a set of frame buffers for the camera input frames in twoseparate banks may be employed. Thus, the system may switch back andforth between two input buffers; while one frame is being processed, thenext frame may be loaded via the PPI/DMA channels.

Core A may also perform background subtraction, thresholding, andblob-finding (i.e., locating possible LED “blobs” in the image), as willbe discussed in greater detail below. Because of latencies involved inmultiple accesses to the same SDRAM bank, data may be transferred fromSDRAM to L1 cache via DMA channels in order to process image datafaster.

The processor can access L1 cache at the system clock speed; therefore,even though it takes some time to transfer data via DMA, performing theprocessing on L1 cache may be significantly faster. One line (752pixels) of data may be transferred at a time into L1 cache, using two L1data buffers when transferring lines via DMA; while one line is beingprocessed, the DMA transfers the next line. The purpose of the buffer,like the input buffers for the entire image frame through the PPI, maybe to minimize the wait time by utilizing hardware memory transfers(i.e., DMA) that do not lock up the processor. On each pixel, backgroundsubtraction may be performed with a reference frame pixel. The referenceframe is updated periodically every few seconds. After backgroundsubtraction, a threshold is used to determine which pixels are examinedfurther in the blob-finding routine. The threshold may adjust manually,by noting at what distances we can distinguish LEDs without bleedingfrom intensities that are too bright in combination with changing theaperture size (thus allowing more or less light into the camera sensor).Alternatively, the threshold may be set automatically to adjust for theaperture size and the threshold used.

In an alternate configuration of a computer or computation device, asdepicted in FIG. 14 b, the computation device 144 may comprise a PC 39having connected thereto the electromagnetic radiation detector (sensor)22 through cable 23. The previously described calculations may beperformed using software stored on or accessible by the PC 39. Suchsoftware may comprise an application programming interface (API) whichmay be exportable to any other PC. Control components, such as radiocontroller 33 may also comprise an independent API. The PC 39 may outputinformation through a cable 35 to a signal converter box 36 forconverting the information to a form transmittable by the radiocontroller 33 and the transmitter antenna 34. Similar data processingtechniques, as discussed above, may also be used in this configuration.

Having provided some exemplary context for use of the system and methodpresented initially in this disclosure, a more detailed discussion ispresented below, with respect to FIGS. 14 c, 1, and 2, of the system andmethod for determining the orientation of an object.

In the embodiment of FIG. 14 a or 14 b, the computation device 144 mayinclude one or more modules for carrying out the method described withrespect to FIG. 11 and more particularly with respect to FIGS. 1 and 2below. Accordingly, as shown in FIG. 14 c, the computation device 144can include an image capture module 160, an image analyzing module 162,a position calculating module 164, and a control development module 166.Each of these modules or components thereof, can include software or aportion thereof, hardware or a portion thereof, or a combination ofsoftware and hardware adapted to perform the associated method. It isalso noted that each module or component thereof can be combined oroverlapped with or combined with modules or components performing othertasks in the process. In some embodiments, this overlap or combinationmay include tasks or steps adjacent to one another in a process, but inother embodiments, the tasks and steps may not be adjacent one another.Moreover, any module or component thereof may or may not be included inthe system depending on the nature of the system desired. Additionally,the computation device 144 or any module or component thereof can eachinclude an input and output module adapted to receive or sendinformation from or to, respectively, other devices, modules, orcomponents. As such, these input and output modules can include physicalports or connection to a bus where the input or output module is of thehardware type. Other types of input and output hardware can be used. Inthe case of software based input and output modules, these can includelines of code causing a processor to step or jump from one location toanother or an application programming interface, for example. Othertypes of software based input and output can also be used.

The modules and components thereof can be located within the computationdevice 144 in one or more of the locations shown in FIGS. 14 a and 14 bfor a given configuration. For example, in the case of a module whereall or a portion of it is software, the software can be located, forexample, in the memory 29, for being accessed by the processor 27. Inother embodiments, the processor 27 can include the software. In thecase of a module where all or a portion of the module is hardware, forexample, the hardware may be a circuit board in communication with thecomputation device 144 for access by the processor 27. Those of skill inthe art will understand and appreciate the several configurationsavailable for using software, hardware, or a combination thereof toprovide a module.

With regard to the image capture module 160, this module can be adaptedto control the detection device 142 such that images of the vehicle canbe captured. For example, this module can include a shutter control andother controls associated with activating the detection device 142 tocapture an image. The capture module can include an initial detectioncomponent that continuously or intermittently activates the detectiondevice to determine whether a vehicle has come into view of thedetection device. Upon recognition of a vehicle, the initial detectioncomponent may activate the detection device. In the active mode, thedetection device may capture images at a certain frequency. To this end,the image capture module may include a timing component that compares anelapsed time since the previous image capture process to a desiredperiod and actuates the detection device when the elapsed time reachesthe desired period. In addition, the image capture module can include ashut down component that deactivates the detection device when a vehicleis no longer in range.

With regard to the image analyzing module 162, this module can beadapted to apply filtering techniques to an image or electronic recordthereof. As such, the image filtering module can perform the imageprocessing portion of step 11 as shown in FIG. 11. More particularly,for example, with regard to FIG. 2, the image filtering module 162 mayinclude a background subtraction component adapted to adjust the imagefor background noise as described above and with respect to FIG. 2below. The image filtering module may also include a threshold imagecomponent, a component labeler, a centroid calculating component, and aLED isolator component. Each of these components can include acombination of software and/or hardware adapted to perform the steps ofFIG. 2 as described below.

With regard to the position calculating module 164, this module can beadapted to determine the position and orientation of a vehicle from thetwo-dimensional representation of the vehicle received from thedetection device 142 and based on the known configuration of positionindicators 128 on the vehicle. As such, the position calculating modulecan be configured to perform the method steps described with respect tomethod step 12 of FIG. 11 and more particularly, the detailed portionsof this step as shown in FIG. 1 described below. As such, the positioncalculating module 164 can include an assumption application component,a processing component, and a result component. Each of these componentscan include a combination of software and/or hardware adapted to performthe steps depicted in FIG. 1 as described below.

With regard to the control development module 166, it is first notedthat this module can be located within the computation device 124 inaddition to or in an alternative to being located within the computationdevice 144. In either or both cases, the control development module canbe adapted to compare the calculated position of the vehicle to thedesired position of the vehicle and provide vehicle control componentcommands for controlling the trajectory or direction of travel of thevehicle. In the case of a UAV, these commands can include aileron,rudder, elevator, and power commands. In other embodiments, the controldevelopment module 166 can be adapted to develop commands forcorresponding vehicle control components. As such, the controldevelopment module 166 can include a plurality of command componentsadapted for development of commands particular to a given controlcomponent of the vehicle. For example, in the case of a UAV, a commandcomponent may be provided for each control component. That is, themodule 166 may include a aileron command component, a rudder commandcomponent, and elevator command component, and a power commandcomponent. In the case of a ground operated vehicle, these components ofthe control development module may include a steering command component,a power command component, and a braking component, for example.

With continued reference to the computer or computation device 144, FIG.2 shows a more detailed method of displaying an image 11, as originallyshown in FIG. 11. Having captured the image (41), the backgroundsubtraction component can subtract the background (that which excludesthe detected point sources) (42) using a reference image or any otherknown technique. Then, a threshold image component may create athreshold image (43) from the brightest remaining pixels. The pointsources remaining on the image may then be digitally labeled buy acomponent labeler with their respective two dimensional (x,y)coordinates (44). In some embodiments, the centroids of the pointsources, if they appear larger than one pixel, may be calculated (45) bya centroid calculating component. Thereby, the LEDs or other pointsources of electromagnetic radiation may be mathematically isolated incoordinate space (46), the positions of which may be used to calculateattitude and position (47), and transmit such positional information tothe control system on the UAV or further perform control instructioncalculations (48).

With particular reference to procedure (43) and thus the functionalityof the threshold image component, one particular known method ofthresholding is the “peak and valley” method. First, a histogram istaken of the intensity values of the image. Then, the threshold ischosen based on the deepest valley (least frequent intensity) betweenthe two peaks (most occurring intensities) in the histogram. Other knownmethods include erosion and dilation. With particular reference toprocedure (44) and thus the functionality of the component labeler,labeling may be accomplished in accordance with any known technique,including that described in “A linear-time component-labeling algorithmusing contour tracing technique,” by Chang et al. With particularreference to procedure (45) and thus the functionality of the centroidcalculating component, centroids may be calculated according to thefollowing “Center of Mass” equation:

$X_{center} = \frac{\sum\limits_{i = 1}^{N}x_{i}}{N}$

Wherein N equals the number of pixels. Such summation may be done inparallel with the procedure (44) for efficiency. Alternatively,centroids may be calculated using a “Bounding Box Approximation”equation:

$X_{center} = \frac{X_{\max} + X_{\min}}{2}$

Wherein Xmax and Xmin are the maximum and minimum pixel locations,respectively. Alternatively other methods may be used such as subpixelinterpolation and dithering to further increase accuracy.

Referring now to FIG. 1, a more detailed chart is shown relating tocalculating the position of an object 100, previously referred to asmethod step 12 on FIG. 9. As previously discussed, positionalinformation may be determined based on the detected positions ofposition indicators 128, and the position of those position indicators128 on the exterior of the vehicle. Such calculation may be made in anymanner known to those of skill in the mathematical arts. In someembodiments, the mathematical calculations may comprise linearapproximations. As depicted in FIG. 1, such a linear approximation maygenerally comprise identifying the reference points (50), decoupling thepoints into 3 orthogonal planes (51) (one for each dimension of movementin space), calculating the angles on the planes, based on the pointcoordinates and the known configuration of the points on the vehicle(52), recombining the three linear dimensional approximations into athree-dimensional orientation and position (53), and transmitting suchinformation to a vehicle control algorithm (54). This control algorithmmay be located in the control system on the vehicle or in thecomputation device connected to the detection device. The assumptionapplication component of the position calculating module 164 can allowfor decoupling the points into the three orthogonal planes by applyingboundary assumptions. The processing component can then calculate theangles on each of the three orthogonal planes, and the results componentcan recombine the three linear dimensional approximations into asolution defining the three dimensional orientation and position of thevehicle.

With particular regard to calculating the position of a vehicle inprocedures 51-53 above, some additional information regarding thebehavior of a three dimensional object in free space can be provided.The behavior of a 3 dimensional object in free space can often occurabout a centralized point and the centralized point is most often thecenter of mass. Free space can be defined as a medium which is uniformlyunrestrictive in all directions such as air, space, water, etc. Themotion of an object being limited to motion about a centralized pointcan allow for decoupling of the objects orientation into threeorthogonal planes intersecting at the centralized point or the center ofmass as noted in FIG. 1. This can occur through the use of referencepoints such as point sources associated with the orientation of theobject. Where the reference points are not positioned so as to becoaxial to any single axis, the orientation of the object can bedetermined. This determination can be most accurate when the referencepoints are further from the center of mass.

In the case of using three reference points, the range of rotations ofthe object can be more limited and efforts to determine the threedimensional orientation from an arbitrary position will still yieldmultiple solutions. However, where the variables being used to solve forthe position are limited, the solution can be obtained more quickly andwithout multiple solutions. For example, bounding conditions in the caseof an aircraft conducting terminal guidance for landing can be based onthe knowledge of the orientation bounds of the aircraft. In the case ofthree reference points for the aircraft landing scenario, it can beassumed that the aircraft will not exceed +/−90 degrees of yaw inrelation to a detection device and further that it will not be invertedon approach. It is noted, however, that even if these bounds areexceeded, there are control methods can be implemented to determine theorientation by observing the behavior of the object in subsequentframes. That is, for example, if the orientation calculation leaves theoption for an upright and an inverted orientation and the airplanereacts in a downward direction due to a control command causing theelevator to create upward motion, the aircraft can be then found to beinverted. However, these assumptions regarding yaw and an uprightapproach allow for solving for the position with a single image ratherthan images over time. Additionally, it is possible to use morereference points and other methods such as using methods to individuallydistinguish each marker through frequency modulation or using wavelengthfiltering.

In cases other than aircraft other assumptions can be made. For example,in the case of an object that is not in free space having a boundedbarrier such as an object sitting on the ground, the orientationbehavior can be different. This can further simplify the orientationcalculation. For example, if the ground surface being encountered isgenerally flat (e.g., a floor of a building) the orientation can bebounded by the ground or a floor. In these cases a more simplifiedapproach can include breaking the analysis into two orthogonal planeswhich are orthogonal to the grounding plane thus being simpler than thethree orthogonal plane approach noted in FIG. 1.

In the case of any vehicle control situation, one set of assumptions canrelate to the dimensions and characteristics of the vehicle beingcontrolled. For example, where reference points are positioned on thevehicle, the reference points can be placed in known positions relativeto the center of mass thereby allowing determination of the vehicleorientation based on these reference point locations and orientation.Additionally, in cases where the currently disclosed methods are used ina sensing and avoiding context, for example, the goal may includecontrolling the behavior of a vehicle where a detection device ispositioned on the vehicle. In these circumstances, the detection devicemay be able to sense or see other objects without knowing theirdimensions or characteristics and yet plot a trajectory for the vehicleto avoid the objects.

Accordingly, a linear approximation may comprise a bounded (usingboundary assumptions) linear calculation using a Taylor seriesexpansion. As discussed and will be appreciated by those skilled in theart, the minimum number of data points required to approximate thepositional orientation, or “pose”, of a three dimensional object that isfree to move and rotate in three dimensions and about three axesrespectively, is three points. In order to achieve the greatestpositional accuracy, these points may be as far from the center ofgravity (CG) of the object. In some embodiments, these points can alsobe coaxial to the axes of rotation. However, in other embodiments thepoints can be mathematically transformed to points falling on the axesas long as all three do not coexist on a single axis of rotation.

As previously discussed, there are several methods which exist tocalculate a three dimensional pose. One computational difficulty thatmay be encountered is that there is always at least one more unknownvariable than there are equations. Using linear approximations to solvefor one unknown variably allows the remaining equations to be solved ina traditional manner, thereby evening the number of unknown variables tothe number of equations required to be solved.

In order to use such a linear approximation, several mathematicalboundary assumptions may be made. Generally, the fewer reference pointsthere are, the more bounded the conditions may need to be for a solutionto be available. Additionally, the analysis time can be greater wherefewer boundary conditions are known or assumed and the time to determinea solution can be a factor in situations such as landing an aircraft,whereas, other situations such as analyzing a stationary object may notbe as concerned with time. In the latter case, multiple images may beused and/or fewer boundary conditions may be assumed.

As eluded to above, the assumptions can be based on the situationinvolving landing of a UAV. Alternatively, these assumptions may beapplied to the control of any vehicle in two or three dimensions. In thecase of an aircraft, three non-collinear reference points are sufficientwith the below boundary conditions to determine an orientation andposition.

In one embodiment, the assumption application component of the positioncalculating module can focus the scope of the solution to the linearequations by applying the following assumptions. First, it may beassumed that the airplane will be approaching the detection device 142from the front. That is, the UAV can be programmed to approach a landingarea from a given direction and the detection device 142 can bepositioned to pickup UAV's as they approach. Second, it may be assumedthat the airplane will be oriented right side up with a roll angle lessthan 90 degrees to either side. This assumption is based on knowledge ofthe UAV flight capabilities as well as general assumptions regardingtheir general attitude status as they approach a landing area. Third, itmay be assumed that the actual dimensions of the UAV are known as wellas the location and distances from the CG of the 3 reference points.This requires that the dimensions and orientations of the positionindicators 128 be placed in particular locations relative to one anotherand in particular locations relative to the plane and further that thisinformation be input into the computer 144 or computation device 124.Fourth, it may be assumed that because the reference points on the wingare close to being co-axial with the CG, then the only transformationthat affects their perceived distance is yaw. Reference points may alsobe mathematically transformed from other positions, not on or near anaxis of rotation, to positions on the wing. It is therefore also assumedthat airplane pivots about its CG. These assumptions are based onknowledge of general airplane construction and flight behavior. Fifth,it may be assumed that positional angles of the detected positionindicators 128 will be calculated in relation to the display image planeof the detection device, and not in relation to “real world”coordinates. This is due to the fact that the display image plane is notreally a plane but a bounded section of a sphere. Therefore, the displayimage plane changes as the position of the aircraft changes in relationto the camera. It will only change as a two dimensional approximationdue to the up, down, left, and right changes of the aircraft, but notforward and backwards.

These assumptions help establish the boundary conditions of thepositional calculations and allow for more quickly determining theposition of the UAV. Similar assumptions can be made for other vehiclesdepending on the nature of the vehicle and the conditions within whichthe vehicle is being used. For example, for a UGV, assumptions relatingto the vehicle 100 being upright and within a certain range of rollangle could be assumed, etc. It is also noted that the assumptionapplication component may or may not be provided depending on the natureof the system. That is, where a particular system is configured for usein a particular application, the system may be loaded with a set oflinear equations, or other three dimensional processing analysis, thathas already been limited by a list of assumptions similar to thoselisted above. In this embodiment, the processing component of theposition calculating module may be loaded with a bounded set of linearequations, or other bounded three dimensional processing analysis,applicable to a particular application.

Having applied the assumptions, the processing component of the positioncalculating module can solve the linear equations to determine certainaspects of the three dimensional orientation and position. For example,in computing the position of a UAV in accordance with the presentdisclosure, the aircraft may first be mathematically “un-yawed” in orderto determine the distance between the wingtips as detected by thedetector. Once this distance is calculated, the range of the aircraftmay be calculated. Then, once the range is known, this variable may beused with the standard linear equations, discussed above, in order tosolve for the aircraft position.

In particular, the yaw of the aircraft may be calculated using thefollowing equation:

${Yaw} = {{\tan^{- 1}\frac{a}{D/2}} = {\sin^{- 1}\frac{a}{D^{\prime}/2}}}$

Wherein D is the observed distance between wingtip reference points, D′is the actual distance between the wingtips, and ‘a’ is the observeddistance between the tail reference point and the center point betweenthe wingtips. For a generalized three dimensional problem, D can be thedistance between two reference points which are either coaxial ormathematically transformed to be coaxial, and ‘a’ can be the observeddistance to a third non-coaxial point.

Then, the range can be calculated using the following equation:

$D^{\prime} = {{focallength} \times \frac{ActualWingspan}{Range}}$

Wherein focal length is determined by the lens chosen for theelectromagnetic radiation detection device, as discussed above.

With yaw and range known, the remaining variables to be solved forinclude roll angle and pitch. Roll angle may be calculated using atrigonometric identity, based on the yaw-corrected wingtip points.Specifically:

$\theta = {\arctan \frac{\Delta \; y}{\Delta \; x}}$

Wherein theta is the roll angle, and x and y are the corrected wingtipcoordinates. Furthermore, pitch may be calculated using the known centerpoint between the wingtips, and the coordinates of the detected pointsource located on the tail or vertical stabilizer. Specifically:

${PitchAngle} = {\sin^{- 1}\frac{2F}{D^{\prime}}}$

Wherein F is the distance from the tail to the center point of the wing.

The results component of the position calculating module can thencombine these results to define the orientation and position of thevehicle. Once the position and orientation of the UAV has beendetermined using the above described equations and calculation methods,position or control information may be developed and transmitted to theUAV, and the control system on the UAV may make appropriate controlinputs to the directional controls of the UAV to achieve or maintain alocalizing course to the desired point, for example, the landing areawhere the detection device has been positioned. This process may becontinually repeated and in this manner, a UAV may be autonomouslycontrolled to the point of landing, so as to enable the UAV to be usablefor subsequent applications/missions.

With specific reference now to the directional control system, asembodied in a UAV, one objective may be to achieve and maintain anacceptable glide slope for the UAV descent which will result in a safeand successful approach to landing. In some applications, it may bedesirable for the glide slope to be configured so as to allow the UAV toclear a vertical wall of approximately 12 feet at a range ofapproximately 500 feet. With the assumption that the components of thesecond sub-system are placed on flat ground, a minimum glides slope of3.4 degrees is required to safely clear the wall in this manner.However, an excessively steep glide slope may result in a verticalvelocity that would cause stress upon the UAV at touchdown. Therefore, aglide slope of approximately between 3.4 degrees to 15 degrees may bedesirable, and more particularly a glide slope of 6 degrees may bedesirable. In other embodiments, steeper or less steep glide angles canbe selected depending on the conditions and surrounding necessary toland the UAV safely and without damage. In some embodiments, the glideangle can be adjusted as the UAV approaches a landing area so as tofeather the approach and provide for a softer landing. The glide anglemay therefore be configurable to allow for precision landing at anypoint in front of the detection device and within the detection range ofthe detection device.

As previously mentioned, there are four control surfaces that exist onthe UAV witch determine the UAV's three dimensional trajectory.Referring once again to FIG. 5, the ailerons 106 on the back of eachwing mainly affect the bank angle. Their movements may be tied together,so as the left aileron rotates down the right aileron rotates up atexactly the same rate. Thus, the left aileron 106 angle may always bethe negative of the right aileron 106 angle with respect to the wing.The elevator 102 on the back of the horizontal stabilizer mainly affectsthe UAV's pitch, moving the nose up or down with respect to the UAV'scenter of gravity. The rudder 104 on the back of the vertical stabilizerprimarily affects the yaw of the UAV. Lastly, the powerplant 108 affectsoverall velocity.

The directional control system may comprise four separate parallelclosed loop systems, each controlling individual control surfaces of theairplane; ailerons, elevator, rudder, and powerplant. Each system mayhave both inner and outer loop components running in parallel which arethen output to the control surface as a weighted sum. This approach tocontrolling the UAV flight may optimize control for optical sensing.

More particularly, aileron controls may be a weighted summation of bankerror based on a constant desired bank of zero degrees, horizontalvelocity on the display image plane displacement, and an integration ofdesired bank error. Elevator controls may be based on a desired pitch ofthe UAV relative to the radial position vector from the center of theimage plane to the normal plane of the aircraft. This may give result inpitch that varies the true pitch of the aircraft with vertical positionon the display image plane. Rudder controls may also be a weightedsummation of the following components: the UAV's yaw relative to theradial position vector from the center of the display image plane to thenormal plane of the UAV, the integration of the yaw error, and theproduct of the horizontal velocity vector in the image plane with theaircraft horizontal displacement in the image plane and the calculatedaircraft range. Throttle controls may be a weighted sum of the verticaldisplacement of the UAV in the image plane and the vertical velocity ofthe aircraft in the image plane. Other directional control algorithmsare known in the art, and may be employed in connection with thedirectional control system in alternative embodiments.

Referring again to FIGS. 14 a and 14 b, a control stick 40 may beoperably connected to the computer or computation device 144. A controlstick 40 may be required where remote manual operation of the UAV isdesired during certain portions of flight. The control stick 40 motionsmay be electronically sent to the computer or computation device 144,indicating desired control changes in the UAV. Alternatively, duringautonomous control, the computer or computation device 144 may generateits own control or position instructions/information, as previouslydiscussed. The computer or computation device 144 may be operablyconnected to a transmitter 146 in a manner, for example, as shown ingreater detail in FIGS. 14 a-b. The transmitter 146 transmits a radio orother electronic signal 41 comprising the aforementioned position orcontrol information. Such information in the signal 41 is receivable bya receiver 122 located in the first sub-system 120 of the UAV. Thereceiver 122 may be operably connected to the directional control system126 of the UAV, which may comprise, for example, various actuator/cableassemblies, servos, hydraulics and air/fuel mixture regulators, amongothers.

In one particular embodiment, the transmitter 146 may be a Futaba6EX-PCM radio system. Such system is a 72 MHz radio system that usesPulse Code Modulation. It sends information via a binary coded signal(the bit length being determined by the number of channels) to thereceiver 122, followed by a 16 bit checksum. Pulse Code modulation maybe desirable as the form of transmission because it is less prone tosignal noise or error, although it will be appreciated that any form oftransmission may be used in accordance with the present disclosure.

As will be appreciated by those skilled in the art, closed-loop feedbackcontrol systems may have an inherent latency between detection andresponse. Such latency may cause instability in the system. In selectingthe particular components of the system as shown in FIG. 1 for use witha particular application, the following considerations may be taken intoaccount which may reduce latency. 1) Employing Pulse Position Modulationtransmissions as opposed to Pulse Coded Modulation; 2) using fewerchannels; 3) using digital servos; or 4) using a 2.4 GHz spread spectrumradio (e.g., a Futuba 2.4 GHz spread spectrum radio system).

With continued reference to the directional control system 126, in orderto control the control component positions, e.g. servo positions, on theUAV (which subsequently control the UAV's movements through theailerons, elevator, rudder, and throttle), a Futaba-specific PulsePosition Modulated (PPM) signal may be sent through the trainer port ofour Futaba radio transmitter (or other similar signal in embodiments notusing Futuba radio systems). The PPM signal may be an approximately 0 to5 Volt digital signal with the following format: 1) An approximately 9ms high synchronizing pulse. 2) A low pulse lasting for approximately400 μs. 3) Up to 8 channels with the following format: a high pulselasting approximately from 0.680 ms to 1.52 ms, with approximately 1.12ms being at a neutral position, indicating the servo position of thatparticular channel, followed by a low pulse of 400 μs. A timer interruptwith a period of 10 μs may be used to output the desired PPM signalthrough a output pin on the BF561 (or similar component of embodimentsusing a computation device other than the BF561). If any signal noise isexperienced during such transmissions, shielded wires or copper foil maybe employed on the electrical components of the system in order tomitigate such noise.

Although the present disclosure has been described with reference tovarious embodiments, persons skilled in the art will recognize thatchanges may be made in form and detail without departing from the spiritand scope of the invention. The techniques of this disclosure may beembodied in a wide variety of devices or apparatuses. Any components,modules, or units have been described to emphasize functional aspectsand does not necessarily require realization by different hardwareunits, etc.

Accordingly, the techniques embodied/described herein may be implementedin hardware, software, firmware, or any combination thereof. Anyfeatures described as modules or components may be implemented togetherin an integrated logic device or separately as discrete butinteroperable logic devices. If implemented in software, the techniquesmay be realized at least in part by a computer-readable mediumcomprising instructions that, when executed, performs one or more of themethods described herein. The computer-readable medium may compriserandom access memory (RAM) such as synchronous dynamic random accessmemory (SDRAM), read-only memory (ROM), non-volatile random accessmemory (NVRAM), electrically erasable programmable read-only memory(EEPROM), FLASH memory, magnetic or optical data storage media, and thelike.

If implemented in software, the software code may be initially stored ona computer readable medium, and may be executed by one or moreprocessors, such as one or more digital signal processors (DSPs),general purpose microprocessors, an application specific integratedcircuits (ASICs), field programmable logic arrays (FPGAs), or otherequivalent integrated or discrete logic circuitry. The term “processor,”as used herein may refer to any of the foregoing structure or any otherstructure suitable for implementation of the techniques describedherein. In addition, in some aspects, the functionality described hereinmay be provided within dedicated software modules or hardware modulesconfigured for encoding and decoding, or incorporated in a combinedvideo codec. Also, the techniques could be fully implemented in one ormore circuits or logic elements.

Many other aspects of this disclosure will become apparent from theteaching below. Nothing in this disclosure should be construed as anyadmission regarding prior art or known systems. Any discussion ofbackground material is provided for context, and does not necessarilymean that such background material was known, or that problems akin tobackground material were known.

1. A system for determining an orientation and position of an objectfrom a single two dimensional image of the object, the systemcomprising: a computation device, comprising: an input module adapted toreceive data defining a single two dimensional image; an image analyzingmodule configured to receive the data and analyze the single twodimensional image to determine a two dimensional orientationrepresentative of a three dimensional orientation and position; aposition calculating module configured to receive the two dimensionalorientation from the image analyzing module and determine the threedimensional orientation and position of the object; and an output moduleadapted to send information relating to the three dimensionalorientation and position of the object.
 2. The system of claim 1,wherein the image analyzing module comprises: a background subtractioncomponent configured to reduce interference associated with the datadefining the two dimensional image; a threshold image componentconfigured to create a threshold image from brightest pixels; and acomponent labeler configured to assign coordinates to selected portionsof the two dimensional image.
 3. The system of claim 2, wherein theimage analyzing component further comprises a centroid calculatingcomponent configured to define a centroid of one or more pixels of thesingle two dimensional image.
 4. The system of claim 1, wherein theposition calculating module further comprises a processing componentconfigured to process a series of linear equations to determine thethree dimensional orientation and position of the object.
 5. The systemof claim 4, wherein the series of linear equations is a Taylor series ofequations.
 6. The system of claim 4, wherein the position calculatingmodule further comprises an assumption application component configuredto apply boundary conditions to the series of linear equations therebysimplifying the processing thereof.
 7. The system of claim 1, furthercomprising a situational data component accessible by the positioncalculating module, wherein situational data is stored for use insimplifying the determination of the three dimensional orientation andposition.
 8. The system of claim 7, wherein the situational datacomprises boundary assumptions relating to the range of expectedorientations of the object defined by a range of rotation angles aboutaxes passing through the center of mass of the object.
 9. The system ofclaim 8, wherein the expected orientations relate to operational limitsand conditions of the object.
 10. The system of claim 8, wherein therange of rotation angles includes a range of angles about a longitudinaldirection of travel of the object.
 11. The system of claim 8, whereinthe range of rotation angles includes a range of angles about one ormore directions transverse to the direction of travel.
 12. The system ofclaim 7, wherein the situational data is adjustable based on the objectand conditions for which the orientation and position are beingdetermined.
 13. The system of claim 7, wherein the situational dataincludes relationship information between the object and a positionindicator associated with the object.
 14. The system of claim 1, whereinthe computation device further comprises an image capturing moduleconfigured to control an image detection device.
 15. The system of claim14, wherein the image capturing module further comprises: an initialdetection component configured to activate a detection device; a timingcomponent configured to control the frequency of detection deviceactuation; and a shut down component configured to deactivate thedetection device.
 16. A method for determining an orientation andposition of an object from a single two dimensional image of the object,the method comprising: receiving image data and storing the image datain a computer readable storage medium, the image data including a singletwo dimensional depiction of the object; and using a computation devicehaving one or more modules for accessing the image data and determiningthe orientation and position of the object, the determining comprising:analyzing the image data to determine a two dimensional orientation thatis representative of a three dimensional position and orientation of theobject; and performing a three dimensional analysis limited by boundaryconditions to determine the three dimensional orientation and positionof the object.
 17. The method of claim 16, wherein analyzing the imagedata further comprises labeling one or more portions of the image datawith two dimensional coordinates.
 18. The method of claim 16, furthercomprising applying boundary conditions to limit the variablesassociated with the three dimensional position and orientation of theobject, wherein the boundary conditions relate to the range of expectedorientations of the object defined by a range of rotation angles aboutaxes passing through the center of mass of the object.
 19. The system ofclaim 16, wherein the expected orientations relate to operational limitsand conditions of the object.
 20. The system of claim 18, wherein therange of rotation angles includes a range of angles about a longitudinaldirection of travel of the object.
 21. The system of claim 18, whereinthe range of rotation angles includes a range of angles about one ormore directions transverse to the direction of travel.
 22. The system ofclaim 18, wherein the boundary conditions further relate to a knownrelationship between the position indicator orientation and the objectorientation.
 23. The method of claim 17, wherein analyzing the imagedata further comprises: filtering background noise out of the twodimensional representation; and creating a threshold image.
 24. Themethod of claim 16, wherein performing the three dimensional analysisincludes processing a system of linear equations.